Enable CPU/XPU native and ipex path #1628
Conversation
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
|
I am cleaning the CPU and XPU tests, process 50% |
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
| quant_state.blocksize, | ||
| quant_state.shape, | ||
| quant_state.dtype, | ||
| ) |
There was a problem hiding this comment.
Is there reason why this change can't be in bitsandbytes/backends/cpu/ops.py?
There was a problem hiding this comment.
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
|
Hi @matthewdouglas . Please check if there are anything missed before merging. We can discuss the tests in this issue #1637 |
Signed-off-by: jiqing-feng <[email protected]>
bitsandbytes/cextension.py
Outdated
| if not ipex_cpu: | ||
| logger.warning( | ||
| "The installed version of bitsandbytes was compiled without IPEX support. " | ||
| "You can install ipex by running `pip install intel_extension_for_pytorch`to get better performance if you use the Intel CPU.", | ||
| ) |
There was a problem hiding this comment.
This seems like extra noise that we'd want to avoid.
Something to point out is that we still plan to ship libbitsandbytes_cpu in our wheels, so for most users, it's going to load a CPU, CUDA, or eventually ROCm or Metal library and we'll hit this logging line. At most we should really only raise this warning when:
- We're on a platform with IPEX CPU support. My understanding is this is limited to Linux x86-64.
- We expect the user to be using CPU, i.e. no CUDA, XPU, or MPS accelerators available.
On torch >= 2.6 we could just usetorch.accelerator.is_available()and on older versions I think we can overlook privateuse1 backends like HPU or Ascend NPU. - There's some expectation of IPEX being beneficial. We don't want to prompt users to install it if e.g. it needs AVX512 or AMX support to be effective. This is something I can't speak to directly but defer to Intel folks to determine.
Any other thoughts @Titus-von-Koeller ?
There was a problem hiding this comment.
You were right. I also agree that the log should only exist if no devices like cuda/xpu are available and the CPU is an Intel product.
There was a problem hiding this comment.
I have changed it, please review again. Thanks!
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
|
Hi @matthewdouglas . You can see some CPU ops are implemented by pure python, which works for all devices. Could we move these pure python implementations to the default folder? As some of these ops could be reused by XPU. |
Signed-off-by: jiqing-feng <[email protected]>
jiqing-feng
changed the title
|
Hi @matthewdouglas . I moved pure pytorch ops to the default folder and it works well. The CIs are all passed. Please continue to review this PR. |
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
matthewdouglas
merged commit aaa71d7
into
bitsandbytes-foundation:main
This PR enables ipex and other optimizations including:
Also, it fixed the parameter patch for cpu.
It could pass all transformers tests
After this PR merged, I will update the installation guide.
@matthewdouglas @Titus-von-Koeller